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DETAILED ACTION 

This Office Action has been issued in response to the amendments filed on 
December 24, 2007. Claims 1-27 are pending with claims 1, 3, 5, 6, 8, 10, 11, 15-17, 
21, and 24 amended. 

Response to Arguments 

1 . Applicant's arguments with respect to claims 1,19, 23, and 26 have been 
considered but are moot in view of the new ground(s) of rejection. 

2. Applicant's arguments filed December 24, 2007 with respect to claims 6, 11 , 16 
have been fully considered but they are not persuasive. 

As per claims 6, 1 1 , and 1 6, applicant argues that none of the references teach 
or suggest "creating an independent speech recognition model based on the first set of 
training data and the second set of training data if the differences in model information 
between first speech recognition model and the second speech recognition model is 
insignificant." However, examiner respectfully disagrees given that Chandrasekar et al. 
teaches that clusters with insignificant differences are merged (Col. 10, lines 59-61 and 
65-66). The act of merging the cluster results in the creation of a new different cluster 
that had not existed prior to the merging. The result of creating a new model based on 
two to other models equates to the result of merging two models to form one model. 
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Claim Objections 

3. Claim 6 is objected to because of tine following informalities: Claim 6, lines 1 -2, 
first claims a "system" for generating speech recognition models and then claims that 
the "method" comprises. Examiner assumes that this is a mere error, and that the claim 
meant to cite "the system comprising." Appropriate correction is required. 

Claim Rejections - 35 USC § 101 

4. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

5. Claims 1-27 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

With respect to claims 1, 6, 16, and 17 the methods and systems claimed consist 
solely of mathematical operations without some practical application. By generating 
speech recognition models, as opposed to recognition models (as previously 
presented), does not give practical application to the claims. If the "acts" of a claimed 
process manipulate only numbers, abstract concepts or ideas, or signals representing 
any of the foregoing, the acts are not being applied to appropriate subject matter. Thus, 
a process consisting solely of mathematical operations, i.e., converting one set of 
numbers into another set of numbers, does not manipulate appropriate subject matter 
and thus cannot constitute a statutory process. 

With respect to claims 1 1 and 24, applicant claims a "computer program product" 
(emphasis added). There is no description or definition found in applicant's disclosure 



Application/Control Number: 10/649,909 Page 4 

Art Unit: 2626 

regarding the computer program product. This subject matter is not limited to that which 
falls within a statutory category of invention because it is not limited to a process, 
machine, manufacture or a composition of matter. This is a practical application in the 
technical arts, however the computer program product and computer readable program 
codes as claimed are simply functional descriptive material, and thus a computer 
program per se. 



Claim Rejections - 35 USC § 103 

6. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

7. Claim 6, 11 , and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chang et al. (US Patent 6,567,776) in view of Chandrasekar et al. (US Patent 
6,578,032). 

As per claims 6 and 16, Chang et al. teach a system for generating speech 
recognition models, the method comprising: 

a first speech recognition model based on a first set of training data, the first set 
of training data originating from a first set of common entities (cluster model 202, Col. 2, 
lines 35-41 , also Fig. 2 shows the two speaker cluster models 202 and 204, each of the 
speaker cluster models having substantially similar characteristics); and 

a second speech recognition model based on a second set of training data, the 
second set of training data originating from a second set of common entities (cluster 
model 204, Col. 2, lines 35-41 , also Fig. 2 shows the two speaker cluster models 202 
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and 204, each of the speaker cluster models having substantially similar 
characteristics). 

However, Chang et al. does not specifically mention 

a processing module configured to create an independent speech recognition 
model based on the first set of training data and the second set of training data if the 
difference in model information between first speech recognition model and the second 
speech recognition model is insignificant. 

Conversely, Chandrasekar et al. teach 

a processing module configured to create an independent speech recognition 
model based on the first set of training data and the second set of training data if the 
difference in model information between first speech recognition model and the second 
speech recognition model is insignificant (Col. 10, lines 59-61 and 65-66, For cluster A 
and cluster C a difference between dusters has been determined to be insignificant. As 
a result cluster A is merged with cluster C forming a newly merged cluster). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of determining models and creating an 
independent model as taught by Chandrasekar et al. for Chang et al.'s system because 
Chandrasekar et al.'s invention automatically analyzes a text string and either updates 
an existing cluster or creates a new cluster (Col. 2, lines 2-4). 

As per claim 1 1 , Chang et al. teach a computer program product embodied in 
computer memory comprising: 
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computer readable program codes coupled to the computer memory for 
generating speech recognition models, the computer readable program codes 
configured to cause the program to: 

receive a first speech recognition model based on a first set of training data, the 
first set of training data originating from a first set of common entities (cluster model 
202, Col. 2, lines 35-41, also Fig. 2 shows the two speaker cluster models 202 and 204, 
each of the speaker cluster models having substantially similar characteristics); and 

receive a second speech recognition model based on a second set of training 
data, the second set of training data originating from a second set of common entities 
(cluster model 204, Col. 2, lines 35-41 , also Fig. 2 shows the two speaker cluster 
models 202 and 204, each of the speaker cluster models having substantially similar 
characteristics). 

However, Chang et al. does not specifically mention 

determine a difference in model information between the first speech recognition 
model and the second speech recognition model; and 

create an independent speech recognition model based on the first set of training 
data and the second set of training data if the difference in model information is 
insignificant. 

Conversely, Chandrasekar et al. teach a cluster A and a cluster C for which a 
difference between clusters has been determined to be insignificant. As a result cluster 
A is merged with cluster C forming a newly merged cluster (Col. 10, lines 59-61 and 65- 
66). 
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It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the features of determining a difference between models and 
creating an independent model as taught by Chandrasekar et al. for Chang et al.'s 
method and computer program product because Chandrasekar et al.'s invention 
automatically analyzes a text string and either updates an existing cluster or creates a 
new cluster (Col. 2, lines 2-4). 

8. Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Chang et 
al. (US Patent 6,567,776) In view of Chandrasekar et al. (US Patent 6,578,032) and 
further in view of Newman et al. (US Patent 6,151,575). 

As per claim 1, Chang et al. teach a method for generating speech recognition 
models, the method comprising: 

receiving a first speech recognition model based on a first set of recorded 
phonemes training data, the first set of training data originating from a plurality of female 
speakers (cluster model 202, Col. 2, lines 35-41 and Col. 4, lines 53-60. According to 
Col. 2, lines 36-40, speaker clusters 202 and 204 have substantially similar 
characteristics, also from Col. 4, lines 53-60, the speech data used for the models come 
from female and male speakers. Therefore, it would have been obvious to conclude 
that the substantially similar characteristics of cluster model 202 come from same 
gender speakers, in this case females.); and 

receiving a second speech recognition model based on a second set of recorded 
phonemes training data, the second set of training data originating from a plurality of 
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male speakers cluster model 204, Col. 2, lines 35-41 and Col. 4, lines 53-60. According 
to Col. 2, lines 36-40, speaker clusters 202 and 204 have substantially similar 
characteristics, also from Col. 4, lines 53-60, the speech data used for the models come 
from female and male speakers. Therefore, it would have been obvious to conclude 
that the substantially similar characteristics of cluster model 204 come from same 
gender speakers, in this case males.). 

However, Chang et al. does not specifically mention 

determining a difference in model information between the first speech 
recognition model and the second speech recognition model; and 

creating a gender-independent speech recognition model based on the first set 
of recorded phonemes training data and the second set of recorded phonemes training 
data if the difference in model information is insignificant. 

Conversely, Chandrasekar et al. teach a cluster A and a cluster C for which a 
difference between clusters has been determined to be insignificant. As a result cluster 
A is merged with cluster C forming a newly merged cluster (Col. 10, lines 59-61 and 65- 
66). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the features of determining a difference between models and 
creating an independent model as taught by Chandrasekar et al. for Chang et al.'s 
method and computer program product because Chandrasekar et al.'s invention 
automatically analyzes a text string and either updates an existing cluster or creates a 
new cluster (Col. 2, lines 2-4). 
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However, neither Chang et al. nor Chandrasekar et al. specifically mention 
the training data to be composed of recorded phonemes. 
Conversely, Newman et al. teach 

the training data to be composed of recorded phonemes (Col. 3, lines 50-52, Col. 
7, lines 3-5, and Col. 8, lines 35-44). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of the training data being composed of 
recorded phonemes as taught by Newman et al. for Chang et al.'s method, as modified 
by Chandrasekar et al. because Newman et al. provides for generating a source- 
adapted model for use in speech recognition (Col. 2, lines 28-29) wherein sources of 
speech data include particular speakers or groups of related speakers (Col. 2, lines 44- 
45). 

9. Claims 2-4 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Chang et al. (US Patent 6,567,776) in view of Chandrasekar et al. (US Patent 
6,578,032) and Newman et al. (US Patent 6,151,575) as applied to claim 1 above, and 
further in view of Kanevsky et al. (US Patent 6,529,902). 

As per claim 2, Chang et al., as modified by Chandrasekar et al. and Newman et 
al., teach the method according to claim 1 , but they do not specifically disclose whether 
the model information is insignificant is based on a threshold model quantity. Kanevsky 
et al. teaches the Kullback-Leibler distance between any two topics is at least h, where 
h is some sufficiently large threshold (Col. 5, lines 9-11). Further, Kanevsky et al. 
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teaches using Kullbacl^-Leibler distance, one can cliecl^ wliicli pairs of topics are 
sufficiently separated from eacli otiier. Topics tliat are close in this metric could be 
combined together (Col. 12, lines 44-47). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the feature of a threshold model quantity as taught by 
Kanevsky et al. for Chang et al.'s method, system, and computer program product as 
modified by Chandrasekar et al. because Kanevsky et al. provides an improved 
language modeling for off-line automatic speech decoding and machine translation (Col. 
2, lines 50-52). 

As per claim 3, Chang et al., as modified by Chandrasekar et al. and Newman et 
al., teach the method according to claim 1 , but they do not specifically disclose that 
determining the difference in model Information includes calculating a Kullback-Lelbler 
distance between the first speech recognition model and the second speech recognition 
model. Kanevsky et al. teaches that for two different sets, one can define a Kullback- 
Lelbler distance using the frequencies of the sets. [With the distance] one can check 
which pairs of topics are sufficiently separated from each other. Topics that are close in 
this metric could be combined together (Col. 12, lines 42-47). 

As per claim 4, Chang et al., as modified by Chandrasekar et al. and Newman et 
al., and in further view of Kanevsky et al., teach the method according to claim 3, 
wherein whether the model information is insignificant is based on a threshold Kullback- 
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Leibler distance quantity (Kanevsky's Col. 5, lines 9-1 1 , the Kullback-Leibler distance 
between any two topics is at least h, where h is some sufficiently large threshold, also 
they teach (Col. 12, lines 44-47) that while using the Kullback-Leibler distance, one can 
check which pairs of topics are sufficiently separated from each other, and that topics 
that are close in this metric could be combined together). 

10. Claims 7-9 and 12-14 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chang et al. (US Patent 6,567,776) in view of Chandrasekar et al. (US Patent 
6,578,032) as applied to claims 6, 1 1 , and 16 above, and further in view of Kanevsky et 
al. (US Patent 6,529,902). 

As per claims 7 and 12, Chang et al., as modified by Chandrasekar et al., teach 
the system and computer program product according to claims 6 and 1 1 , but they do 
not specifically disclose whether the model information is insignificant is based on a 
threshold model quantity. Kanevsky et al. teaches the Kullback-Leibler distance 
between any two topics is at least h, where h is some sufficiently large threshold (Col. 5, 
lines 9-11). Further, Kanevsky et al. teaches using Kullback-Leibler distance, one can 
check which pairs of topics are sufficiently separated from each other. Topics that are 
close in this metric could be combined together (Col. 12, lines 44-47). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the feature of a threshold model quantity as taught by 
Kanevsky et al. for Chang et al.'s method, system, and computer program product as 
modified by Chandrasekar et al. because Kanevsky et al. provides an improved 
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language modeling for off-line automatic speech decoding and machine translation (Col. 
2, lines 50-52). 

As per claim 8, Chang et al., as modified by Chandrasekar et al., teach the 
system according to claim 6, but they do not specifically disclose that determining the 
difference in model information includes calculating a Kullback-Leibler distance between 
the first speech recognition model and the second speech recognition model. Kanevsky 
et al. teaches that for two different sets, one can define a Kullback-Leibler distance 
using the frequencies of the sets. [With the distance] one can check which pairs of 
topics are sufficiently separated from each other. Topics that are close in this metric 
could be combined together (Col. 12, lines 42-47). 

As per claim 13, Chang et al., as modified by Chandrasekar et al., teach the 
computer program product according to claim 1 1, but they do not specifically disclose 
that determining the difference in model information includes calculating a Kullback- 
Leibler distance between the first model and the second model. Kanevsky et al. teaches 
that for two different sets, one can define a Kullback-Leibler distance using the 
frequencies of the sets. [With the distance] one can check which pairs of topics are 
sufficiently separated from each other. Topics that are close in this metric could be 
combined together (Col. 12, lines 42-47). 
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As per claims 9 and 14, Chang et al. as modified by Chandrasekar et al. and in 
further view of Kanevsky et al. teach the system, and computer program product 
according to claims 8 and 13, wherein whether the model information is insignificant is 
based on a threshold Kullback-Leibler distance quantity (Kanevsky's Col. 5, lines 9-11, 
the Kullback-Leibler distance between any two topics is at least h, where h is some 
sufficiently large threshold, also they teach (Col. 12, lines 44-47) that while using the 
Kullback-Leibler distance, one can check which pairs of topics are sufficiently separated 
from each other, and that topics that are close in this metric could be combined 
together). 

1 1 . Claim 5 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over Chang et 
al. (US Patent 6,567,776) in view of Chandrasekar et al. (US Patent 6,578,032) and 
Newman et al. (US Patent 6,151,575) as applied to claim 1 above, and further in view of 
Wark (US 2003/0231775). 

As per claim 5, Chang et al., as modified by Chandrasekar et al. and Newman et 
al., teach the method of claim 1 , but they do not disclose the first speech recognition 
model, second speech recognition model, and gender-independent speech recognition 
model are Gaussian mixture models. 

However, Wark teaches multiple class models defined as Gaussian mixture 
models (paragraph [0135], the Gaussian mixture model Xc with c=1,2, C, where C is 
the number of class models). 
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It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have defined the models as Gaussian mixture models as taught by 
Wark for Chang et al.'s method, system, and computer program product as modified by 
Chandrasekar et al. because Wark's invention relates generally to audio signal 
processing and, in particular, to the classification of semantic events in audio streams 
(paragraph [0001]). 

12. Claims 10 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Chang et al. (US Patent 6,567,776) in view of Chandrasekar et al. (US Patent 
6,578,032) as applied to claims 6 and 1 1 above, and further in view of Wark (US 
2003/0231775). 

As per claims 10 and 15, Chang et al. as modified by Chandrasekar et al., teach 
the system and computer program product of claims 6 and 11 , but they do not disclose 
the first speech recognition model, second speech recognition model, and gender- 
independent speech recognition model are Gaussian mixture models. 

However, Wark teaches multiple class models defined as Gaussian mixture 
models (paragraph [0135], the Gaussian mixture model Xc with c=1,2, C, where C is 
the number of class models). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have defined the models as Gaussian mixture models as taught by 
Wark for Chang et al.'s method, system, and computer program product as modified by 
Chandrasekar et al. because Wark's invention relates generally to audio signal 
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processing and, in particular, to the classification of semantic events in audio streams 
(paragraph [0001]). 

13. Claims 17-18, 21-22 and 24-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Wark (US 2003/0231775) in view of Verma et al. (US 
2002/0174086), and further in view of Chang et al. (US Patent 6,567,776), 
Chandrasekar et al. (US Patent 6,578,032), and Newman et al. (US Patent 6,151,575). 

As per claims 17, 21 , and 24, Wark teaches a method, system, and computer 
program product for recognizing speech from an audio stream originating from one of a 
plurality of data classes, the method comprising: 

receiving a current feature vector of the audio stream (modules 202, 206, and 
208 from Fig. 2, and Paragraph [0139].); and 

computing a current vector probability that the current feature vector belongs to 
one of the plurality of data classes (Paragraph [0139] describes equation (30) which 
calculates the model score between the clip feature vectors ft of the segment and one of 
the C object models (C is the number of class models as defined in paragraph [0135]). 
The model score is the summing of the log of the probability of the feature vector fi 
belonging to class Xc). 

However, Wark does not specifically mention 

computing an accumulated confidence level that the audio stream belongs to one 
of the plurality of data classes based on the current vector probability and on previous 
vector probabilities; 
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weighing class models based on the accumulated confidence; and 

recognizing the current feature vector based on the weighted class models. 

Conversely, Verma et al. teaches an accumulated confidence level that the data 
stream belongs to one of the plurality of data classes based on the current vector 
probability and on previous vector probabilities (paragraph [0020], cumulative mean Hi 
of the sample confidence Lij over a large number of samples is used to measure the 
overall discrimination capability of the classifier, also paragraphs [0018] and [0019] 
describe the sample confidence Lij as the log-likelihood of the kth most likely class such 
that the lykS form order statistics, that is Iijl>lij2>.. >lijn) ; weighing class models based on 
the accumulated confidence (paragraph [0024], a weight wij is assigned to each 
classifier as a function of the overall confidence Hi and the sample confidence Lij); and 
recognizing the current feature vector based on the weighted class models (paragraph 
[0024], once weights wij for each classifier are known, each incoming sample can be 
classified in a class by calculating the combined log-likelihood for each class). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the feature of a cumulative confidence level as taught by 
Verma et al. for Wark's method, system, and computer program product because 
Verma et al. provides method, system, and computer program product that improves 
the classification accuracy of particular decision fusion applications such as medical 
imaging, biometric verification, signature or fingerprint verification, robot vision, speech 
recognition, image retrieval, expert systems, etc (paragraph [0002]). 

However, neither Wark nor Verma et al. specifically mention 
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wherein the plurality of data classes include a female speech recognition model 
based on recorded phonemes originating from plurality of female speakers, a male 
speech recognition model based on recorded phonemes originating from plurality of 
male speakers, and a gender- independent speech recognition model based on 
recorded phonemes originating from plurality of both female and male speakers having 
insignificant differences in information. 

Conversely, Chang et al, in view of Chandrasekar et al., and further in view of 
Newman et al. teach 

wherein the plurality of data classes include a female speech recognition model 
based on recorded phonemes originating from plurality of female speakers, a male 
speech recognition model based on recorded phonemes originating from plurality of 
male speakers, and a gender- independent speech recognition model based on 
recorded phonemes originating from plurality of both female and male speakers having 
insignificant differences in information (see rejection for claim 1). 

As per claims 18, 22, and 25, Wark, as modified by Verma et al., Chang et al, 
Chandrasekar et al, and Newman et al., teach the method, system, and computer 
program product according to claims 17, 21 and 24 above, wherein computing the 
current vector probability includes estimating an a posteriori class probability for the 
current feature vector. (Wark's equation (30) clearly makes use of p(fi/Xc) or the 
probability that feature vector fi belongs to class Xc. This conditional probability is 
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equivalent to an a posteriori probability because it is the probability of the feature vector 
given that it belongs to a certain class). 

14. Claims 20 and 27 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Wark (US 2003/0231775) in view of Verma et al. (US 2002/0174086), and further 
in view of Chang et al. (US Patent 6,567,776), Chandrasekar et al. (US Patent 
6,578,032), and Newman et al. (US Patent 6,151,575) as applied to claims 17 and 24 
above, and further in view of Catchpole (US 2005/0251390). 

As per claims 20 and 27, Wark, as modified above, teach the method and 
computer program product according to claims 17 and 24 above, but they do not 
specifically disclose the method and computer program product further comprising 
determining if another feature vector is available for analysis. 

However Catchpole teaches a lexical tree processor that attempts to read the 
next feature vector from the feature vector buffer and if it's not available an error occurs, 
if the vector is available the tree processor reads the feature vector from the buffer 
(paragraph [0044], first lines). 

It would have been obvious to one having ordinary skill in the art at the time of 
the invention to have used the feature of determining if another feature vector is 
available for analysis as taught by Catchpole for Wark's method and system, as 
modified above because Catchpole provides a circuit that performs parallel processing 
of speech parameters (paragraph [0004]). 
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1 5. Claims 1 9, 23, and 26 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Wark (US 2003/0231775) in view of Verma et al. (US 
2002/0174086), and further in view of Chang et al. (US Patent 6,567,776), 
Chandrasekar et al. (US Patent 6,578,032), and Newman et al. (US Patent 6,151,575) 
as applied to claims 1 7, 21 , and 24 above, and further in view of Eisner et al. (Bayesian 
Analysis of U.S. Hurricane Climate-2001). 

As per claims 19, 23 and 26, Wark, as modified above, teach the method, 
system, and computer program product of claims 17, 21, and 24, respectively, but they 
do not specifically mention wherein computing the accumulated confidence level further 
comprising weighing the current vector probability more than the previous vector 
probabilities. 

However, Eisner et al. teach 

wherein computing the accumulated confidence level further comprising 
weighing the current vector probability more than the previous vector probabilities 
(Section 6. Refinements, First paragraph lines 13-23). 

It would have been obvious to a person having ordinary skill in the art at the time 
of the invention to have used the feature of weighing the current vector probability more 
than the previous vector probabilities as taught by Eisner et al. for Wark's method, 
system, and computer program product, as modified above, because Eisner et al. 
provides a Bayesian probability approach which gives a rational and coherent 
foundation for using all available information while explicitly accounting for differences in 
uncertainty (Eisner et al.. Section 4, lines 7-10). This approach is well known in the field 
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of probability and statistics. It would have also been obvious to a person having 
ordinary skill in the art at the time of the invention to give more weight to current 
information, which carries more importance, than to past information. The accumulation 
of these probabilities permits small or big changes in time to be noticeable which is also 
helpful for determining future probabilities. 



Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to NATALIE LENNOX whose telephone number is 
(571 )270-1649. The examiner can normally be reached on Monday to Friday 9:30 am - 
7 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 

supervisor, Richemond Dorvil can be reached on (571)272-7602. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

ML 03/17/2008 
/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



